REPORT  DOCUMENTATION  PAGE 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  lor  reviewing  instructions,  searching  e 
the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  coition  of 'information,  including  suggestions 
Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  I 


AFRL-SR-AR-TR-oa. 


1.  AGENCY  USE  ONLY  (Leave  blank} 


I  2.  REPORT  DATE 


4.  TITLE  AND  SUBTITLE 

Competent  Probabilistic  Model  Building  Genetic  Algorithms 


3.  REPORT  TYPE  AND  DATETCOVtKcir - - 

1  Feb  00  -  30  Nov  02 


5.  FUNDING  NUMBERS 

F49620-00- 1-0163 


6.  AUTHOR(S) 

David  Goldberg 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 
Department  of  General  Engineering 
The  University  of  Illinois  at  Urbana-Champaign 
117  Transportation  Building 
104  S.  Mathews  Avenue 

Urbana.  IL  61801 _ 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

AFOSR/NM 

4015  Wilson  Blvd,  Room  713 
Arlington,  VA  22203-1954 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 

F49620-00- 1-0163 


12a.  DISTRIBUTION  AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE,  DISTRIBUTION  UNLIMITED 


13.  ABSTRACT  (Maximum 200  words) 

AFOSR  Grant  No.  F49620-00-1-0163  has  come  to  a  close  with  substantial  progress  on  all  fronts.  In  particular,  the  Bayesiar 
Optimization  Algorithm  has  been  extended  to  the  important  class  of  hierachically  difficult  problems.  Progress  has  also  been 
made  in  a  number  of  related  competent  GAs  and  efficiency  enhancement  techniques  as  well  as  in  the  application  of  these 
techniques. 


20030818  OH 


17.  SECURITY  CLASSIFICATION 
OF  REPORT 


18.  SECURITY  CLASSIFICATION 
OF  THIS  PAGE 


19.  SECURITY  CLASSIFICATION 
OF  ABSTRACT 


15.  NUMBER  OF  PAGES 


16.  PRICE  CODE 


i20.  LIMITATION  OF  ABSTRAC . 


Standard  Form  298  (Rev.  2-89  (EG) 

Prescribed  by  ANSI  Std.  239.18 

Designed  using  Perform  Pro,  WHS/DIOR,  Oct  94 


Progress  and  Final  Report 

Competent  Probabilistic  Model  Building 
Genetic  Algorithms 
AFOSR  Grant  No.  F49620-00-1-0163 

01  December  1999  to  30  November  2002 


David  E.  Goldberg 

Department  of  General  Engineering 
IlliGAL  Report  No.  X0016  (Limited  Circulation) 
July  2003 


Department  of  General  Engineering 
University  of  Illinois  at  Urbana-Champaign 
117  Transportation  Building 
104  South  Mathews  Avenue 
Urbana, IL  61801 


Progress  &  Final  Report  for 

Competent  Probabilistic  Model  Building 
Genetic  Algorithms 
AFOSR  Grant  No.  F49620-00-1-0163 

1  December  1999  to  30  November  2002 

David  E.  Goldberg 

Department  of  General  Engineering 
The  University  of  Illinois  at  Urbana-Champaign 
117  Transportation  Building 
104  S.  Mathews  Avenue 
Urbana,  IL  61801 


Abstract 

AFOSR  Grant  No.  F49620-00-1-01630  (Competent  Probabilistic  Model  Building  Genetic 
Algorithms)  has  come  to  a  close  with  substantial  progress  on  all  fronts.  In  particular,  the 
Bayesian  Optimization  Algorithm  has  been  extended  to  the  important  class  of  hierarchically 
difficult  problems.  Progress  has  also  been  made  in  a  number  of  related  competent  GAs  and 
efficiency  enhancement  techniques  as  well  as  in  the  application  of  these  techniques. 


1  Objectives 

The  objectives  of  AFOSR  Grant  No.  F49620-00-1-01630  (Competent  Probabilistic  Model  Building 
Genetic  Algorithms)  were  as  follows: 

1.  Develop,  implement,  and  enhance  two  probabilistic  model  building  GAs:  BOA  (Bayesian 
optimization  algorithm)  and  ecGA  (extended  compact  GA). 

2.  Test  BOA  and  ecGA  on  rigorous  theoretical  function  test  suite. 

3.  Extend  BOA  and/or  ecGA  to  hierarchically  difficult  problems. 

4.  Extend  BOA  and/or  ecGA  to  alternative  codings. 

5.  Apply  the  developed  algorithms  to  two  problems  of  Air  Force  interest. 

Significant  progress  has  been  made  on  each  fronts.  This  progress  is  briefly  reviewed  in  the  next 
section. 
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2  Status  of  Effort 


The  project  has  come  to  a  close  with  key  progress  in  each  of  the  objective  areas  above.  Key  areas 
of  progress  are  highlighted  below. 

hBOA.  A  major  task  of  the  project  was  to  advance  the  cause  of  probabilistic  model-building  ge¬ 
netic  algorithms ,  a  particular  type  of  competent  GA  that  solve  hard  problems,  quickly,  reliably,  and 
accurately.  Progress  in  this  area  over  the  course  of  the  project  has  been  substantial  and  included 
particular  concentration  on  the  hierarchical  Bayesian  optimization  algorithm  (hBOA).  Martin  Pe- 
likan’s  groundbreaking  dissertation  on  hBOA  is  arguably  the  most  notable  achievement  of  the 
project.  Recall  that  simple  BOA  builds  a  Bayesian  network  model  of  the  best  points  in  each  gener¬ 
ation,  using  this  model  in  place  of  crossover,  mutation,  and  other  genetic  operators  to  speculate  on 
the  best  points  in  the  search  space.  Because  of  the  generality  of  the  Bayes  net  assumption,  BOA 
can  quickly  search  for  optimal  solutions  under  more  general  conditions  than  can  a  linkage-oriented 
solver.  Hierarchical  BOA  takes  the  original  scheme,  combines  it  with  niched  selection  and  chunking 
techniques  to  form  a  technique  that  quickly  solves  hierarchically  difficult  problems. 

The  technique  has  been  theoretically  and  empirically  shown  to  be  sub-  or  nearly  quadratic  over 
a  very  large  class  of  problems — to  what  Herbert  Simon  called  the  class  of  nearly  decomposable 
problems.  This  effort  has  resulted  in  a  the  filing  of  a  patent  through  the  University  of  Illinois  Office 
of  Technology  Management. 

hBOA  on  maxSAT  and  spin  glasses.  We  have  published  work  on  applying  hBOA  to  satisfiabil¬ 
ity  problems  and  nearest  neighbor  spin  glasses  in  two  and  three  dimensions.  Of  course,  worst  case 
satisfiability  problems  are  intractable,  but  if  substructure  exists,  hBOA  appears  to  be  able  to  detect 
it  and  exploit  it.  In  spin-glass  problems,  hBOA  is  competitive  with  problem-specific  heuristics. 

Multiobjective  hBOA.  Nazan  Khan  wrote  a  useful  MS  thesis  extending  BOA  and  hBOA  to 
difficult  multiobjective  problems.  Simply  stated,  a  standard  selection  technique  of  evolutionary 
multiobjective  optimization  (EMO),  NSGA-2,  was  added  to  Pelikan’s  hBOA,  thereby  creating  a 
very  powerful  multiobjective  solver. 

Hybrid  BOA  and  ES.  First  steps  were  taken  to  use  BOA  in  real-coded  problems  by  using  a 
discrete  BOA  solver  in  concert  with  self-adaptive  evolution  strategies.  The  approach  was  successful 
in  combining  the  global  search  capability  of  BOA  with  the  local  rapidity  of  an  ES. 

Other  model  builders  invented.  Other  model-building  techiques  were  invented  as  part  of  this 
effort.  In  particular,  standard  techniques  of  machine  learning  such  as  ID3  have  been  adapted  to  the 
task  (Llora  &  Goldberg,  2003)  as  have  techniques  inspired  by  design  structure  matrices  or  DSMs 
(Yu,  Yassine,  &  Goldberg,  2003).  The  latter  effort  resulted  in  a  patent  disclosure  for  both  the  GA 
and  organizational  implications  of  our  work, 

Knjazew’s  OmeGA.  Dimitri  Knjazew,  a  visiting  scholar  from  Dortmund,  Germany,  and  the  PI 
developed  OmeGA,  the  ordering  messy  genetic  algorithm  for  solving  hard  problems  over  permuta¬ 
tions.  Using  previously  supported  fast  messy  GA  technology,  the  researchers  have  come  up  with 
a  way  to  solve  difficult  scheduling  and  sequencing  problems  quickly,  reliably,  and  accurately.  This 
work  is  being  published  as  a  book  by  Kluwer  Academic  Publishers. 

Tsutsui’s  real-Coded  linkage  GA  &:  PMBGA.  Professor  Shigeyoshi  Tsutsui  finished  his  visit 
from  Hannan  University  and  his  work  on  two  new  techniques  has  been  published.  Tsutsui  and 
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Pelikan  worked  together  to  carry  the  framework  of  BOA  over  to  real-coded  problems  with  success. 

LLGA  revisted.  Earlier  work  in  the  lab  developed  the  so-called  linkage  learning  GA.  Y.  P.  Chen 
has  revisited  that  work  and  extended  it  by  using  encapsulating  punctuation  marks  to  improve 
building  block  separation  and  mixing.  A  previous  defect  of  the  LLGA  was  its  inability  to  solve 
problems  with  multiple  uniformly  scaled  building  blocks  simultaneously.  To  carry  this  work  forward, 
new  theory  has  been  developed  of  tightness  time  to  predict  when  good  linkage  will  be  achieved. 

Advanced  scalability  theory.  Mixing  is  arguably  the  crucial  step  to  achieve  scalable  GAs. 
Sastry’s  work  (Sastry  &  Goldberg,  2003)  advances  the  state  of  the  art  by  building  principled 
models  of  population  size  and  run  duration  for  one-point  and  other  low-mixing  crossover  operators. 

Time  utilization.  Ravi  Srivastava  completed  an  MS  thesis  on  the  theory  of  time  continuation  and 
its  validation.  This  work  settles  the  great  debate  between  crossover  and  mutation  on  economic  and 
solution-quality  grounds.  Essentially,  the  theory  shows  that  a  combination  of  the  serial  processing 
of  mutation  and  the  parallel  processing  of  mixing  or  crossover  is  optimal  in  most  problems  and  the 
degree  of  balance  is  related  to  the  salience  of  the  building  blocks  or  other  features  of  the  problem 
that  give  it  a  sequential  nature. 

Evaluation  relaxation.  Theses  by  Laura  Albert  and  Kumara  Sastry  add  to  our  knowledge  about 
time-quality  tradeoffs  in  fitness  evaluation.  Sastry’s  work  contrasts  the  temporal  strategies  needed 
in  problems  of  pure  bias  to  spatial  strategies  required  in  problems  of  pure  variance.  Albert’s  work 
applies  these  ideas  to  evaluation  relaxation  in  problems  based  on  the  numerical  solution  of  integral 
equations.  The  key  result  is  that  at  any  given  point  in  a  run,  the  accuracy  of  the  numerical 
integration  should  be  of  the  same  order  as  the  precision  required  of  each  decision  variable,  and 
discretization  is  scheduled  in  a  principled  manner  to  make  this  happen. 

Hybridization.  A  first  analysis  of  the  optimization  of  global-local  search  was  performed  in  1999 
and  Abhishek  Sinha  has  completed  an  MS  thesis  on  this  topic.  The  initial  theory  required  static 
probabilities  of  hitting  a  basin  of  attraction  to  the  local  search.  With  an  adaptive  global  search 
technique  like  a  GA,  those  probabilities  must  be  assumed  to  vary.  Prasanna  Parthasarathy  has 
studied  hybrid  structural  optimization  and  has  developed  a  novel  technique  of  Baldwinian  sharing 
for  such  problems  as  well  as  a  technique  to  save  function  evaluations  by  identifying  basins  of 
attraction  approximately.  Practical  self-adaptive  techniques  were  studied  in  concert  with  Minsker 
and  her  PhD  student  Espinoza. 

Parallelization.  Cantu-Paz’s  work  in  1999  continues  to  inform  our  understanding  of  GA  par¬ 
allelism  in  a  sufficiently  adequate  and  accurate  manner.  A  number  of  items  were  revisited  as  a 
result  of  this  study.  The  issue  of  mult-  versus  single-run  studies  was  analyzed,  and  multiple  inde¬ 
pendent  runs  are  sometimes  advantageous.  Master-slave  parallelism  was  revisited  when  significant 
GA  overhead  exists.  The  standard  formulae  for  optimal  slave  counts  remain  valid,  although  over¬ 
all  expressions  have  been  derived  that  show  the  additional  degradation  to  speedup  in  cases  with 
significant  GA  overhead. 

Time-quality  analysis  of  different  representations.  Franz  Rothlauf  completed  his  dissertation 
investigating  code  redundancy,  locality,  and  scaling.  His  framework  closes  the  loop  on  coding  design 
by  investigating  codes  in  a  quality-time  framework  for  the  first  time.  This  is  a  critical  milestone 
and  will  help  put  the  study  of  representations  on  a  more  scientific  footing  that  has  been  possible 
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until  now. 


Symmetry  and  multimodality  studied.  Clarissa  Van  Hoyweghen  has  completed  her  study  of 
spin  glasses  and  certain  network  problems  with  massive  and  essential  symmetry  and  multimodality. 
Walsh  analyses  have  been  performed  and  these  have  resulted  in  basic  understanding  of  the  challenges 
these  problems  pose  for  genetic  algorithms.  The  problems  studied  to  date  have  succumbed  to  GAs 
augmented  with  effective  nichers  such  as  sharing,  restricted  tournament  selection,  deterministic 
crowding,  and  the  like. 

Application  of  GAs  and  GP  in  material  science.  Working  with  Duane  Johnson  here  at 
Illinois,  Kumara  Sastry  has  used  GAs  and  GP  to  speed  the  modeling  of  advanced  materials  9 
orders  of  magnitude.  This  work  is  continuing. 

Electromagnetic  applications  underway.  Contact  has  been  made  with  researchers  at  Hanscom 
AFB  and  studies  are  currently  underway  using  Matlab  code  written  at  Hanscom  as  well  as  standard 
EM  analysis  procedures  (NEC). 

Books  completed  and  published.  Cantu-Paz’s  Efficient  and  Accurate  Parallel  Genetic  Al¬ 
gorithms  (Kluwer)  was  published  in  1999.  Butz’s,  Anticipatory  Classifier  Systems  (Kluwer)  was 
published  in  2002.  Knjazew’s  OmeGA  (Kluwer)  was  published  in  2002.  The  Pi’s  Design  of  Inno¬ 
vation  (Kluwer)  was  published  in  2002.  Rothlauf’s  Representations  for  Genetic  and  Evolutionary 
Algorithms  (Springer)  was  published  in  2002.  Pelikan  is  publishing  a  volume  from  his  hBOA  dis¬ 
sertation  with  Springer. 

3  Personnel  Supported 

This  section  details  the  individuals  supported  on  these  projects. 

3.1  Faculty  supported 

Professor  David  E.  Goldberg,  the  principal  investigator,  was  supported  during  the  summers  of 
2000-2002. 

3.2  Faculty  or  post-docs  supported  by  or  affiliated  with  the  project 

The  following  is  a  list  of  visiting  faculty  supported  by  or  affiliated  with  the  project.  Unsupported 
afElates  may  have  some  travel  or  incidental  expenses  paid  by  the  project. 

1.  Professor  Matthieu  Domaszewski  (Universite  de  Technologie  de  Belfort-Montbeliard,  sup¬ 
ported  partially  by  AFOSR) 

2.  Professor  Felipe  Padilla  (University  Autonomous  of  Aguascalientes  ,  Mexico) 

3.  Professor  T.  K.  Kurian  (Calicut  Regional  Engineering  College,  India,  supported  partially  by 
AFOSR) 

4.  Professor  Shigeyoshi  Tsutsui  (Hannan  University,  Japan) 

5.  Professor  Pier  Luca  (Politecnico  di  Milano,  Italy) 

6.  Dr.  Xavier  Llora  (Ramon  Llull  University,  Barcelona,  Spain) 
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7.  Dr.  Kei  Ohnishi  (Kyushu  University,  Japan) 

3.3  Graduate  students  supported  &;  affiliated  with  project 

The  following  is  a  list  of  graduate  students  supported  or  affiliated  with  the  project  Unsupported 
affilates  may  have  had  travel  or  incidental  expenses  supported  by  the  project. 

1.  L.  Albert  (UIUC,  supported  by  AFOSR) 

2.  M.  Butz  (University  of  Wuerzburg,  Germany  &  NCSA) 

3.  Y.  P.  Chen  (UIUC,  supported  on  TA) 

4.  A.  Ceroni  (University  of  Florence,  Italy) 

5.  N.  Khan  (UIUC,  NCSA) 

6.  A.  Kosorukoff  (UIUC,  supported  by  UI  Research  Board  &  TEC) 

7.  P.  Parthasarathy  (UIUC,  supported  by  AFOSR  &  NSF) 

8.  M.  Pelikan  (UIUC,  supported  by  AFOSR) 

9.  F.  Rothlauf  (University  of  Bayreuth,  Germany) 

10.  K.  Sastry  (UIUC,  supported  by  AFOSR  &  NSF) 

11.  A.  Sinha  (UIUC,  supported  by  AFOSR) 

12.  R.  Srivastava  (UIUC,  supported  by  NSF) 

13.  T.-L.  Yu  (UIUC,  supported  by  AFOSR) 

14.  K.  Sastry  (UIUC,  supported  by  AFOSR,  Army,  &;  NSF) 

15.  C.  Van  Hoyweghen  (University  of  Antwerp,  Belgium) 

3.4  Undergraduate  students  supported  by  or  affiliated  with  the  project 

The  following  is  a  list  of  undergraduate  research  assistants  who  have  been  supported  by  or  affiliated 
with  the  project: 

1.  L.  Babayeva  (AFOSR) 

2.  J.  Borgerson  (AFOSR  &  NSF) 

3.  C.  Cunningham  (NSF) 

4.  J.  Daly  (NSF) 

5.  K.  Hawley  (AFOSR  &  NSF) 

6.  K.  Kakugawa  (NSF  REU  researcher) 

7.  J.  Leesman  (AFOSR  &  NSF) 

8.  T.  Latoza  (NSF  REU  researcher) 


5 


9.  M.  Lipinski  (AFOSR) 

10.  M.  Perry  (AFOSR) 

11.  N.  Sis  (AFOSR) 

12.  A.  Vaughn  (NSF 

13.  J.  Wilson  (AFOSR) 

4  Publications 

Publications  submitted,  accepted,  and  published  over  the  course  of  the  project  are  presented  in 

this  section. 

4.1  Submitted 

Reed,  P.,  Minsker,  B.  S.,  &  Goldberg,  D.  E.  (2002).  Simplifying  multiobjective  optimization: 
An  automated  design  methodology  for  the  nondominated  sorted  genetic  algorithm-II.  Water 
Resources  Research. 

Sastry,  K.,  Johnson,  D.  D.,  Goldberg,  D.  E.,  &  Bellon,  P.  (2003).  Genetic  programming  for 
multi-timescale  modeling:  A  proof-of-principle  study.  Physical  Review  Letters. 

4.2  Accepted,  Not  Yet  Published 

Butz,  M.  V.,  &  Goldberg,  D.  E.  (in  press).  Generalized  state  values  in  an  anticipatory  learning 
classifier  system.  In  M.  V.  Butz,  Sigaud,  O.,  &  Gerard,  P.  (Eds.),  Anticipatory  behavior  in 
adaptive  learning  systems.  Heidelberg,  Germany:  Physica-Verlag. 

Butz,  M.,  &;  Goldberg,  D.  E.  (in  press).  Bounding  the  population  size  in  XCS  to  ensure  reproduc¬ 
tive  opportunities.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation 
Conference. 

Butz,  M.,  &  Goldberg,  D.  E.  (in  press).  Tournament  selection:  Stable  fitness  pressure  in  XCS. 
GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference. 

Butz,  M.,  Goldberg,  D.  E.,  &  Tharakunnel,  K.  (in  press).  Analysis  and  improvement  of  fitness 
exploitation  in  XCS:  Bounding  models,  tournament  sselection,  and  bilateral  accuracy.  Evolu¬ 
tionary  Computation. 

Cantu-Paz,  E.  &  Goldberg,  D.  E.  (in  press).  Are  multiple  runs  of  genetic  algorithms  better  than 
one.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference. 

Chen,  Y.-P.,  &  Goldberg,  D.  E.  (in  press).  Tightness  time  for  the  linkage  learning  genetic  algo¬ 
rithm.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Conference. 

Chen,  Y.-P.,  &  Goldberg,  D.  E.  (in  press).  An  analysis  of  a  reordering  operator  with  tournament 
selectio  on  a  GA-hard  problem.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference. 
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Espinoza,  F.,  Minsker,  B.  &  Goldberg,  D.  E.  (in  press).  Performance  evaluation  and  population 
reduction  for  a  self-adaptive  hybrid  genetic  algorithm  (SAHGA).  GECCO-2003:  Proceedings 
of  the  Genetic  and  Evolutionary  Computation  Conference. 

Goldberg,  D.  E.  (in  press).  John  H.  Holland,  facetwise  models,  and  economy  of  thought.  In  L. 
Booker,  S.  Forrest,  M.  Mitchell,  &  R.  Riolo  (Eds.),  Perspectives  on  Adaptation  in  Natural  and 
Artificial  Systems.  New  York:  Oxford  University  Press. 

Llora,  X.,  Goldberg,  D.  E.,  Traus,  I.,  &  Bernado,  E.  (in  press).  Accuracy,  parsimony,  and  gen¬ 
erality  in  evolutionary  learning  systems  via  multiobjective  selection.  Advances  in  Learning 
Classifier  Systems. 

Llora,  X.,  &  Goldberg,  D.  E.  (in  press).  Bounding  the  effect  of  noise  in  multiobjective  learning 
classifier  systems.  Evolutionary  Computation. 

Llora,  X.,  &  Goldberg,  D.  E.  (in  press).  Wise  breeding  GA  via  machine  learning  techniques  for 
function  optimization.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computa¬ 
tion  Conference. 

Pelikan,  M.  (in  press).  Hierarchical  Bayesian  optmization  algorithm:  Toward  a  new  generation  of 
evolutionary  algorithms.  Heidelberg,  Germany:  Physica-Verlag. 

Pelikan,  M.,  &  Goldberg,  D.  E.  (in  press).  Hierarchical  BOA  solves  Ising  spring  glasses  and 
MAXSAT.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary  Computation  Confer¬ 
ence. 

Sastry,  K.,  &  Goldberg,  D.  E.  (in  press).  Scalability  of  selectorecombinative  genetic  algorithms 
for  problems  with  tight  linkage.  GECCO-2003:  Proceedings  of  the  Genetic  and  Evolutionary 
Computation  Conference. 

Tharakunnel,  K,  Butz,  M.,  &  Goldberg,  D.  E.  (in  press).  Towards  building  block  propagation  in 
XCS:  A  negative  result  and  its  implications.  GECCO-2003:  Proceedings  of  the  Genetic  and 
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tion.  Evolutionary  Computation,  10(4),  317-344. 

Espinoza,  F.,  Minsker,  B.  S.,  k  Goldberg,  D.  E.  (2003).  Local  search  issues  for  the  application  of 
a  self-adaptive  hybrid  genetic  algorithm  in  groundwater  remediation  design. 

Khan,  N.  (2003).  Bayesian  optimization  algorithm  for  multiobjective  and  hierarchically  difficult 
problems  (MS  thesis).  General  Engineering,  University  of  Illinois  at  Urbana-Champaign,  Ur¬ 
bana,  IL. 
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An  approach  using  stochastic  multi-objective  genetic  algorithms.  ASCE  Environmental  & 
Waste  Resources  Institute  (EWRI). 

Singh,  A.,  Minsker,  B.  S.,  Goldberg,  D.  E.  (2003).  Combining  reliability  and  Pareto  optimality — 
An  approach  using  stochastic  multi-objective  genetic  algorithms.  ASCE  Environmental  & 
Waste  Resources  Institute  (EWRI). 

Sinha,  A.  (2003).  Designing  efficient  genetic  and  evolutionary  algorithm  hybrids  (MS  thesis). 
General  Engineering,  University  of  Illinois  at  Urbana-Champaign,  Urbana,  IL. 


5  Interactions  and  Transitions 

This  section  lists  meeting  participation,  presentations,  and  transitions. 

5.1  Meeting  participation  and  presentation 

All  conference  papers  above  represent  presentations  by  Professor  Goldberg  and  his  students.  Pro¬ 
fessor  Goldberg  gave  dozens  of  tutorials,  keynote  talks,  tutorials,  and  other  invited  seminars  and 
lectures  across  the  country  and  around  the  world. 

5.2  Transitions 

ISGEC.  The  PI  was  founding  chair  (1999)  for  the  International  Society  for  Genetic  and  Evolution¬ 
ary  Computation,  which  sponsors  the  regular  series  of  GECCO  conferences.  The  2003  conference 
was  held  in  Chicago  with  520  attendees. 

Kluwer  Book  Series.  The  PI  is  the  consulting  editor  of  a  book  series  called  the  Kluwer  Series 
on  Genetic  Algorithms  and  Evolutionary  Computation  (http://www.wkap.n1/prod/s/GENA). 

GA  course  online.  Pi’s  GA  course  (GE  485)  is  offered  online  via  the  web  this  semester.  It  is  one 
of  the  most  popular  web  courses  offered  by  the  University  of  Illinois  and  the  National  Technological 
University.  See  the  website  at  http://online.cen.uiuc.edu/webcourses/ge485/index.html. 

Innovation  course  online.  The  Pi’s  book,  The  Design  of  Innovation,  has  been  turned  into  an 
eight-lecture  online  short  course  (http://online.engr.uiuc.edu/shortcourses/innovation/index.html). 

Cooling  system  design  with  Caterpillar.  The  PI  has  completed  a  GA-based  cooling  system 
design  package  for  Caterpillar. 

Telecommunications  problems.  A  visitor  from  Germany,  Franz  Rothlauf  (University  of  Bayreuth) 
and  the  PI  worked  on  solving  some  telecommunications  problems  using  random  keys  codings. 

hBOA  patent  pending.  The  hierarchical  Bayesian  optimization  algorithm  is  the  subject  of  a 
patent  filing  by  the  University  of  Illinois.  One  company  has  already  expressed  interest  in  licensing 
hBOA,  and  negotiations  are  underway. 

Technology  transfer  laboratory  established.  A  rapidly  growing  software  company  in  Israel 
established  a  research  laboratory  in  Champaign  to  help  transfer  the  competence  and  efficiency 
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results  of  this  project  to  their  commericial  software  in  mobile  telecommunications.  Results  to  date 
have  been  promising. 

Knjazew  to  SAP  in  Germany.  Dimitri  Knjazew  who  developed  the  OmeGA  code  has  taken  a 
position  with  SAP  in  Germany  where  he  is  adapting  these  techniques  to  SAP’s  product  line. 

Srivastava  to  Jump  Trading.  Ravi  Srivastava  took  a  job  following  his  MS  with  a  financial  firm, 
Jump  Trading,  to  apply  GAs  to  solving  problems  in  financial  portfolio  and  trading  analysis. 

Spinoffs  in  IT  and  organizational  theory.  GA  research  is  leading  to  spinoffs  of  the  work  in  the 
creation  of  collaborative  IT  systems  using  HBGAs  and  interactive  GAs.  GA-style  theory  is  leading 
to  new  quantitative  models  in  organizational  theory  using  elementary  tradeoffs  (ETOs). 

GA-based  companies.  There  are  growing  numbers  of  GA-based  companies  in  the  US  and  around 
the  world  using  technology  affected  by  my  work.  Optimax,  a  GA-based  scheduling  software  firm 
has  an  integrated  product  for  factory  scheduling;  Optimax  recently  sold  out  for  close  to  $60M  to 
i2,  another  scheduling  software  firm.  The  Schema  Company  in  Israel  has  a  GA-based  software  for 
the  telecommunications  business.  Silicon  Biology  in  Minnesota  has  developed  a  data-entry  software 
package  with  GA-based  technology  at  its  core.  Engineous  Systems  iSIGHT  software  use  GAs  as  a 
base  optimization  engine.  A  center  in  England  has  spawned  a  company  called  Optimal  Solutions 
that  uses  structured  messy  genetic  algorithms  to  optimize  water  pipelines. 

6  New  discoveries,  inventions,  or  patent  disclosures 

hBOA  patent.  A  US  patent  filing  was  made  in  January  2002  for  the  hierarchical  Bayesian 
optimization  algorithm  by  the  University  of  Illinois. 

DSM  GA  and  GAs  for  DSMs.  Design  structure  matrices  have  formed  the  basis  of  a  new 
competent  GA  and  GAs  have  been  used  to  cluster  DSMs.  Both  sides  of  the  DSM  invention  (GAs 
for  DSMs  and  DSMs  for  GAs)  have  been  disclosed  to  the  university  and  a  patent  filing  is  anticipated. 

7  Honors  and  Awards 

This  section  lists  awards  for  the  PI  and  his  students. 

7.1  PI  honors  and  awards 

JSD  Professor.  Professor  Goldberg  was  named  Jerry  S.  Dobrovolny  Distinguished  Professor  of 
Entrepreneurial  Engineering  in  May  2003. 

ISGEC  Senior  Fellow.  Professor  Goldberg  was  named  part  of  the  inaugural  class  of  Senior 
Fellows  for  the  International  Society  for  Genetic  and  Evolutionary  Computation  in  July  2003. 

Other  major  awards.  In  1997,  Professor  Goldberg  was  named  a  Gambrinus  Fellow  during  his 
sabbatical  to  Dortmund,  Germany.  In  1996,  Professor  Goldberg  received  the  ASEE  Wickenden 
Award  for  best  paper  in  the  Journal  of  Engineering  Education  for  his  paper  entitled  “Change  in 
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Engineering  Education:  One  Myth,  Two  Scenarios,  and  Three  Foci.”  In  1995,  Professor  Goldberg 
was  named  an  Associate  of  the  Center  for  Advanced  Study  (CAS)  at  the  University  of  Illinois.  In 
1985,  Professor  Goldberg  was  named  an  NSF  Presidential  Young  Investigator. 

7.2  Student  honors  and  awards 

2002  CSE  fellowship  to  Sastry.  Kumara  Sastry  won  a  2002-2003  Computational  Science  and 
Engineering  fellowship  at  the  University  of  Illinois  at  Urbana-Champaign  for  his  work  applying 
genetic  programming  to  material  science. 

2003  CSE  fellowship  to  Butz.  Martin  Butz  won  a  2003-2004  Computational  Science  and 
Engineering  fellowship  at  the  University  of  Illinois  at  Urbana-Champaign  for  his  work  in  anticipation 
and  genetics-based  machine  learning. 

GECCO-2002  best  paper.  Alex  Kosorukoff  won  best  paper  in  methodology,  philosophy,  and 
pedagogy  (MPP)  at  the  2002  GECCO  for  his  paper  “Evolutionary  Computation  as  a  Form  of 
Organization”  (with  D.  E.  Goldberg). 

GECCO-2003  best  paper.  Martin  Butz  and  Kumara  Sastry  won  best  paper  in  classifiers  systems 
(LCS)  at  the  GECCO-2003  for  their  paper  “Tournament  Selection:  Stable  Fitness  Pressure  in  XCS” 
with  D.  E.  Goldberg). 

GECCO-2002  best  graduate  student  workshop  paper.  Y.-P.  Chen  took  honors  as  best 
graduate  student  workshop  paper  at  GECCO-2002  for  his  work  on  extending  the  linkage  learning 
GA. 
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